Image editing method, image editing system, and storage medium storing program

ABSTRACT

An information processing apparatus acquires a captured image by causing a camera to image a projection region in a position where a center portion of an imaging range at the time when the projection region including a projection target object in a real space in which a projector and the projection target object are disposed, a first guide image being displayed in the projection region by the projector, and a center portion of the first guide image overlap. The information processing apparatus generates a projection image by causing a user to edit the captured image and transmits projection image data representing the generated projection image to the projector.

The present application is based on, and claims priority from JPApplication Serial Number 2022-022674, filed Feb. 17, 2022, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an image editing method, an imageediting system, and a storage medium storing a program.

2. Related Art

In recent years, projection mapping for performing a variety ofperformances by projecting various images from a projector onto anobject having a three-dimensional shape has been spreading. In thefollowing explanation, an object onto which an image is projected from aprojector is referred to as projection target object. To perform theprojection mapping, it is necessary to create a two-dimensional image tobe projected from the projector, that is, a two-dimensional image fromthe view point of the projector. Various techniques concerning thecreation of the two-dimensional image from the view point of theprojector have been proposed. Examples of the techniques include atechnique disclosed in JP-A-2017-192040 (Patent Literature 1). Thetechnique disclosed in Patent Literature 1 acquires a projectionapparatus view point image of a three-dimensional projection target,prepares a three-dimensional model corresponding to the projectiontarget as projection content, and converts the three-dimensional modelinto a two-dimensional image coinciding with the projection apparatusview point image to thereby create a two-dimensional image to beprojected from a projection apparatus.

To perform the projection mapping using the technique disclosed inPatent Literature 1, a user is requested to have expertise such asknowledge concerning a shape of a projection target object seen from theview point of a projector and knowledge concerning a three-dimensionalmodel. However, a user who intends to perform the projection mappingdoes not always have such expertise. The technique disclosed in PatentLiterature 1 has a problem in that a user not having expertise cannoteasily recognize the projection mapping.

SUMMARY

According to an aspect of the present disclosure, there is provided animage editing method including: acquiring a captured image by imaging,with a camera, from a position in a real space where a center portion ofan imaging range of the camera and a center portion of a first imageoverlap, a projection region in the real space including a projectiontarget object, which is a projection destination of an image from aprojector, the first image being displayed in the projection region bythe projector; and transmitting a second image obtained by editing thecaptured image to the projector.

According to an aspect of the present disclosure, there is provided animage editing system including: a camera; and a processing apparatusconfigured to control the camera, the processing apparatus executing:acquiring a captured image by imaging, with a camera, from a position ina real space where a center portion of an imaging range of the cameraand a center portion of a first image overlap, a projection region inthe real space including a projection target object, which is aprojection destination of an image from a projector, the first imagebeing displayed in the projection region by the projector; andtransmitting a second image obtained by editing the captured image tothe projector.

According to an aspect of the present disclosure, there is provided anon-transitory computer-readable storage medium storing a program forcausing a computer to execute: acquiring a captured image by imaging,with a camera, from a position in a real space where a center portion ofan imaging range of the camera and a center portion of a first imageoverlap, a projection region in the real space including a projectiontarget object, which is a projection destination of an image from aprojector, the first image being displayed in the projection region bythe projector; and transmitting a second image obtained by editing thecaptured image to the projector.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration example of an image editingsystem according to an embodiment of the present disclosure.

FIG. 2 is a diagram showing a configuration example of an informationprocessing apparatus.

FIG. 3 is a diagram showing an example of a first guide image.

FIG. 4 is a diagram showing a setting example of a projection targetobject with respect to a projection region.

FIG. 5 is a diagram showing a relation among a captured image, a secondguide image, and a superimposed image.

FIG. 6 is a diagram showing a display example of the superimposed image,a message, and a user interface image.

FIG. 7 is a diagram showing an example of the superimposed image at thetime when a center portion of the first guide image and a center portionof the second guide image substantially coincide.

FIG. 8 is a diagram showing an example of a projection image editedbased on a reference image.

FIG. 9 is a diagram showing a projection example of a projection imageby a projector.

FIG. 10 is a flowchart showing a flow of an image editing methodexecuted by a processing device according to a program.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Technically preferable various limitations are added to an embodimentexplained below. However, embodiments of the present disclosure are notlimited to the embodiment explained below.

1. Embodiment

FIG. 1 is a diagram showing a configuration example of an image editingsystem 1 according to an embodiment of the present disclosure. The imageediting system 1 is an information processing system for editing imagedata representing an image projected from a projector 10 onto aprojection region including a projection target object SC. Theprojection target object SC in this embodiment is a fan, on paper ofwhich a person is drawn. The projector 10 and the projection targetobject SC are set in a room of a user who uses the projector 10. Theroom of the user who uses the projector 10 is an example of the realspace in the present disclosure.

The projector 10 is an apparatus that projects image light correspondingto image data supplied from an image supply apparatus to thereby projectan image represented by the image data. As explained in detail below,the image supply apparatus in this embodiment is an informationprocessing apparatus 20. Although detailed illustration is omitted inFIG. 1 , the projector 10 includes a projection lens, a liquid crystaldriver, a liquid crystal panel, and a light source unit. The liquidcrystal driver drives the liquid crystal panel according to image datasupplied from the image supply apparatus to thereby draw an imagerepresented by the image data on the liquid crystal panel. The lightsource unit includes a light source such as a halogen lamp or a laserdiode. Light from the light source is modulated for each of pixels inthe liquid crystal panel and projected as image light via the projectionlens.

The projection region in this embodiment is a predetermined region ofthe projector 10 in the room. An image is projected from the projector10 onto the projection region, whereby the image is displayed in theprojection region. In the following explanation, the image projected bythe projector 10 is referred to as projection image. Image datarepresenting the projection image is sometimes referred to as projectionimage data. The projection target object SC is included in theprojection region. In this embodiment, projection mapping for decoratingthe projection target object SC is realized by displaying the projectionimage in the projection region. The projection image may be a stillimage or may be a moving image. The projection image in this embodimentis a still image.

As shown in FIG. 1 , the image editing system 1 includes the informationprocessing apparatus 20, a camera 30, and a terminal apparatus 40. InFIG. 1 , the projector 10, the projection target object SC, acommunication network 50, and an image storage apparatus 60 areillustrated besides the image editing system 1.

The information processing apparatus 20 is, for example, a stick-typepersonal computer. The information processing apparatus 20 includes amale connector conforming to a predetermined standard such as the USB(Universal Serial Bus). The projector 10 includes a female connectorcorresponding to the male connector. The male connector of theinformation processing apparatus 20 is inserted into the femaleconnector of the projector 10, whereby the information processingapparatus 20 and the projector 10 are electrically connected. Theinformation processing apparatus 20 communicates with the camera 30 andthe terminal apparatus 40 by radio or wire.

The camera 30 is an apparatus for imaging the projection region. Thecamera 30 is set in the room of the user using, for example, a tripod ina posture in which the optical axis of the camera 30 is directed to theprojection target object SC. In this embodiment, the user can optionallychange a position where the camera 30 is set in the room. The camera 30performs imaging under control by the information processing apparatus20 and outputs image data representing a captured image to theinformation processing apparatus 20. In the following explanation, theimage data representing the captured image is referred to as capturedimage data.

The communication network 50 is an electric communication line such asthe Internet. The image storage apparatus 60 is connected to thecommunication network 50. The image storage apparatus 60 is, forexample, a data server. One or a plurality of material data are storedin the image storage apparatus 60. Specific examples of the image datastored in the image storage apparatus 60 include projection image dataedited by the image editing system 1 and captured image datarepresenting a captured image captured by the camera 30. Since the imagedata are stored in the image storage apparatus 60, the user of theprojector 10 can reuse an edited projection image or a captured image ofthe projection region.

The terminal apparatus 40 is a smartphone used by the user of theprojector 10. As shown in FIG. 1 , the terminal apparatus 40 includes anexternal IF device 410, a display device 420, and an input device 430.The external IF device 410 includes a communication circuit thatcommunicates with the image storage apparatus 60 via the communicationnetwork 50 and communicates with the information processing apparatus20. IF is an abbreviation of Interface. The display device 420 includesa liquid crystal display and a driving circuit for the liquid crystaldisplay. The terminal apparatus 40 displays various images on thedisplay device 420 under the control by the information processingapparatus 20. The input device 430 is a transparent sheet-like pressuresensitive sensor provided to cover a surface region of the displaydevice 420 and receives input operation of the user. The terminalapparatus 40 transmits, via the external IF device 410, to theinformation processing apparatus 20, input operation data indicatinginput operation of the user to the input device 430. Consequently, theinput operation of the user is transmitted to the information processingapparatus 20. An image drawing tool, which is software for generating aprojection image by editing a captured image captured by the camera 30,is installed in advance in the terminal apparatus 40. The image drawingtool corresponds to the editing function in the present disclosure.

As explained in detail below, the information processing apparatus 20performs, according to the input operation to the terminal apparatus 40,imaging of the projection region by the camera 30, editing of projectionimage data corresponding to input operation of the user performed on theinput device 430, and output of the edited projection image data to theprojector 10. The projector 10 projects, onto the projection region, aprojection image represented by the projection image data output fromthe information processing apparatus 20.

FIG. 2 is a diagram showing a configuration example of the informationprocessing apparatus 20. As shown in FIG. 2 , the information processingapparatus 20 includes a processing device 210, an external IF device220, and a storage device 230. The processing device 210 includes aprocessor such as a CPU (Central Processing Unit), that is, a computer.The processing device 210 may be configured by a single processor or maybe configured by a plurality of processors. The processing device 210operates according to a program PA stored in the storage device 230 tothereby function as a control center of the information processingapparatus 20.

The external IF device 220 includes the male connector explained above.In a state in which the male connector is inserted into the femaleconnector of the projector 10 and the information processing apparatus20 and the projector 10 are electrically connected, the external IFdevice 220 outputs, to the projector 10, data or a signal given from theprocessing device 210. The external IF device 220 includes acommunication circuit that communicates with the camera 30 or theterminal apparatus 40.

The storage device 230 is a storage medium readable by the processingdevice 210. The storage device 230 includes, for example, a nonvolatilememory and a volatile memory. The nonvolatile memory is, for example, aROM (Read Only Memory), an EPROM (Erasable Programmable Read OnlyMemory), or an EEPROM (Electrically Erasable Programmable Read OnlyMemory). The volatile memory is, for example, a RAM (Random AccessMemory).

The program PA for causing the processing device 210 to execute theimage editing method of the present disclose is stored in advance in thenonvolatile memory of the storage device 230. The volatile memory of thestorage device 230 is used by the processing device 210 as a work areain executing the program PA.

When detecting the connection of the information processing apparatus 20to the projector 10, the processing device 210 reads out the program PAfrom the nonvolatile memory to the volatile memory and starts executionof the read-out program PA. When detecting connection of the terminalapparatus 40 and the camera 30, the processing device 210 operatingaccording to the program PA functions as a projection controller 210 a,a display controller 210 b, a notifier 210 c, an acquirer 210 d, and anoutput unit 210 e shown in FIG. 2 . The projection controller 210 a, thedisplay controller 210 b, the notifier 210 c, the acquirer 210 d, andthe output unit 210 e shown in FIG. 2 are software modules realized bycausing the processing device 210 to operate according to the programPA. Functions respectively performed by the projection controller 210 a,the display controller 210 b, the notifier 210 c, the acquirer 210 d,and the output unit 210 e shown in FIG. 2 are as explained below.

The projection controller 210 a outputs, to the projector 10, a firstsignal for instructing the projector 10 to project an image indicating aprojection range of the projector 10. The projection range of theprojector 10 is decided according to a direction of the optical axis ofthe projector 10 and an angle of view of the projector 10. In thefollowing explanation, the image indicating the projection range of theprojector 10 is referred to as first guide image GA1 Specific examplesof the first guide image GA1 include a rectangular image uniformlypainted out in a first color such as white and the rectangular imagebordered with a second color different from the first color. The secondcolor in this embodiment is red. However, the second color is notlimited to red. FIG. 3 is a diagram showing an example of the firstguide image GA1 In FIG. 3 , hatching represents the first color and adotted line represents a line drawn in the second color. That is, thefirst guide image GA1 in this embodiment is a rectangular imageuniformly painted out in the first color and bordered with the secondcolor. A region surrounded by the dotted line is the projection range.The first guide image GA1 is an example of the first image in thepresent disclosure. The projector 10 that has received the first signalprojects the first guide image GA1 onto the projection region. The userwho has visually recognized the first guide image GA1 displayed in theprojection region sets the projection target object SC in the projectionrange indicated by the first guide image GA1 as shown in FIG. 4 . InFIG. 4 , drawing of the hatching indicating the first color is omitted.That is, the projection target object SC is included and the first guideimage GA1 is displayed in the projection region.

The display controller 210 b output a second signal for instruction animaging start to the camera 30. When receiving the second signal, thecamera 30 performs imaging at a predetermined period such as a severalmilliseconds interval and outputs captured image data to the informationprocessing apparatus 20. As explained above, the camera 30 is set in theroom of the user in a posture in which the optical axis of the camera 30is directed to the projection target object SC. Accordingly, projectionregion where the first guide image GA1 is displayed and the projectiontarget object SC set in the projection region are reflected in acaptured image GA2 captured by the camera 30.

Every time captured image data is input from the camera 30, the displaycontroller 210 b generates a superimposed image GA4 shown in FIG. 5based on the captured image data. FIG. 5 is a diagram showing a relationamong the superimposed image GA4, the captured image GA2 represented bythe captured image data, and a second guide image GA3. The second guideimage GA3 is an image indicating an imaging range of the camera 30. Theimaging range of the camera 30 is decided according to a direction ofthe optical axis of the camera 30 and an angle of view of the camera 30.The second guide image GA3 in this embodiment is an image of arectangular frame line partitioning the outer edge of the captured imageGA2. In this embodiment, the frame line is colored in a third colordifferent from the first color and different from the second color. InFIG. 5 , the coloring by the third color is represented by an alternatelong and short dash line. The third color in this embodiment is blue.However, the third color is not limited to blue. As shown in FIG. 5 ,the superimposed image GA4 is an image obtained by superimposing thesecond guide image GA3 on the captured image GA2.

As shown in FIG. 6 , the display controller 210 b displays thesuperimposed image GA4 on the display device 420. The display controller210 b displays, on the display device 420, a user interface image GA5for requesting the user to perform input for determining a position ofthe camera 30. As shown in FIG. 6 , the user interface image GA5 in thepresent disclosure is an image of a virtual operation piece.

The notifier 210 c performs notification for requesting the user to movethe camera 30 to a position where a center portion of the first guideimage GA1 and a center portion of the second guide image GA3 overlap.The center portion of the first guide image GA1 means a region on theinner side of a first circle having a predetermined radius and centeringon a center position of the first guide image GA1 in the first guideimage GA1 The center portion of the second guide image GA3 means aregion on the inner side of a second circle having the predeterminedradius and centering on a center position of the second guide image GA3in the second guide image GA3. The center position of the first guideimage GA1 means an intersection of two diagonal lines in the first guideimage GA1 Similarly, the center position of the second guide image GA3means an intersection of two diagonal lines in the second guide imageGA3. The radius of the circles defining the center portions of the firstguide image GA1 and the second guide image GA3 may be set to a preferredvalue by an experiment or the like. The center portion of the firstguide image GA1 and the center portion of the second guide image GA3overlapping means that the distance between the center of the firstcircle and the center of the second circle is smaller than the radiusexplained above.

As explained above, the first guide image GA1 corresponds to theradiation range of the projector 10 and the second guide image GA3corresponds to the imaging range of the camera 30. Therefore, the centerportion of the first guide image GA1 and the center portion of thesecond guide image GA3 overlapping is equivalent to a center portion ofthe projection range of the projector 10 and a center portion of theimaging range of the camera 30 overlapping.

In this embodiment, as shown in FIG. 6 , the notifier 210 c displays, onthe display device 420, a message M1 of words for requesting the user tomove the camera 30 to the position where the center portion of the firstguide image GA1 and the center portion of the second guide image GA3overlap. In this embodiment, the displaying the message M1 on thedisplay device 420 is equivalent to the performing the notification forrequesting the user to move the camera 30 to the position where thecenter portion of the first guide image GA1 and the center portion ofthe second guide image GA3 overlap. The notifier 210 c may perform thenotification by outputting reading voice of the message M1. The user whohas visually recognized the message M1 carries the camera 30 and moveswhile checking an overlapping state of the center portion of the firstguide image GA1 and the center portion of the second guide image GA3through the superimposed image GA4 displayed on the display device 420.According to the movement of the camera 30, the overlapping state of thecenter portion of the first guide image GA1 and the center portion ofthe second guide image GA3 in the superimposed image GA4 displayed onthe display device 420 changes.

The acquirer 210 d determines, based on the superimposed image GA4,whether the size of an overlapping region where the center portion ofthe first guide image GA1 and the center portion of the second guideimage GA3 overlap is equal to or larger than a threshold. The thresholdmay be decided according to the area of the first circle or the secondcircle. Specific examples of the threshold in the aspect include a valueof 60% to 90% of the area of the first circle or the second circle. Whenthe size of the overlapping region is smaller than the threshold, theacquirer 210 d causes the notifier 210 c to continue displaying themessage M1. When the size of the overlapping region is equal to orlarger than the threshold, the acquirer 210 d causes the notifier 210 cto end displaying the message M1. FIG. 7 is a diagram showing a displayexample of the superimposed image GA4 and the user interface image GA5in the case in which the size of the overlapping region is equal to orlarger than the threshold. Since the size of the overlapping region isequal to or larger than the threshold, the message M1 is not displayedin the example shown in FIG. 7 . As shown in FIG. 7 , when the messageM1 is not displayed on the display device 420 any longer, the usertouches the user interface image GA5.

When detecting the touch on the user interface image GA5, first, theacquirer 210 d outputs, to the projector 10, a third signal forinstructing the projector 10 to stop projecting the first guide imageGA1 Subsequently, the acquirer 210 d writes captured image data outputfrom the camera 30 in the volatile memory of the storage device 230 asreference image data representing a reference image serving as areference in editing a projection image. The reference image data isimage data representing the captured image GA2 in the projection regionwhere the first guide image GA1 is not displayed.

The output unit 210 e acquires projection image data representing aprojection image by causing the user to edit the reference image. Inthis embodiment, first, the output unit 210 e transmits the referenceimage data to the terminal apparatus 40 and causes the terminalapparatus 40 to start an image drawing tool. The terminal apparatus 40executing the image drawing tool displays the image represented by thereference image data on the display device 420. The user edits, usingthe image drawing tool, a projection image for decorating the projectiontarget object SC while referring to the position and the shape of theprojection target object SC reflected in the reference image.

FIG. 8 is a diagram showing an example of the projection image editedbased on the reference image. In FIG. 8 , the projection target objectSC reflected in the reference image is represented by a dotted line.Images of stars added with horizontal-line hatching are projectionimages drawn by the user in order to decorate the projection targetobject SC. The horizontal-line hatching in FIG. 8 represents a coloradded to the images of the stars such as yellow. The output unit 210 eacquires image data representing the projection image edited based onthe reference image from the terminal apparatus 40 as the projectiontarget image. The projection image edited based on the reference imageis an example of the second image in the present disclosure. The outputunit 210 e outputs the projection image data to the projector 10. Theprojector 10 projects the projection image represented by the projectionimage data output from the information processing apparatus 20 onto theprojection region. As a result, as shown in FIG. 9 , the images of thestars for decorating the projection target object SC are displayedaround the projection target object SC. Horizontal-line hatching in FIG.9 represents a color added to the images of the stars such as yellow.

The position of the camera 30 at the time when the reference image iscaptured is the position where the center portion of the first guideimage GA1 and the center portion of the second guide image GA3 overlap.The center portion of the first guide image GA1 and the center portionof the second guide image GA3 overlapping means that the center portionof the projection range of the projector 10 and the center portion ofthe imaging range of the camera 30 overlap. The center portion of theprojection range of the projector 10 and the center portion of theimaging range of the camera 30 overlapping means that a direction of theoptical axis of the projector 10 and a direction of the optical axis ofthe camera 30 substantially coincide and an angle of view of theprojector 10 and an angle of view of the camera 30 substantiallycoincide. Therefore, a position and a shape of the projection targetobject SC in the reference image substantially coincide with a positionand a shape of the projection target object SC at the time when theprojection target object SC is viewed from the projector 10.Accordingly, the user can create a projection image matching theposition and the shape of the projection target object SC by drawing aprojection image for decorating the projection target object SC whilereferring to the position and the shape of the projection target objectSC reflected in the reference image.

The processing device 210 operating according to the program PA executesan image editing method shown in FIG. 10 . As shown in FIG. 10 , theimage editing method in this embodiment includes projection controlprocessing SA110, display control processing SA120, notificationprocessing SA130, acquisition processing SA140, and output processingSA150.

In the projection control processing SA110, the processing device 210functions as the projection controller 210 a. In the projection controlprocessing SA110, the processing device 210 causes the projector 10 todisplay the first guide image GA1 by outputting, to the projector 10, afirst signal for instructing the projector 10 to project the first guideimage GA1

In the display control processing SA120, the processing device 210functions as the display controller 210 b. In the display controlprocessing SA120, the processing device 210 outputs a second signal forinstructing imaging to the camera 30. Subsequently, the processingdevice 210 displays, on the display device 420, the superimposed imageGA4 obtained by superimposing the second guide image GA3 on the capturedimage GA2 captured by the camera 30 and the user interface image GA5.

In the notification processing SA130, the processing device 210functions as the notifier 210 c. In the notification processing SA130,the processing device 210 displays the message M1 on the display device420 to thereby perform notification for requesting the user to move thecamera 30 to the position where the center portion of the first guideimage GA1 and the center portion of the second guide image GA3 overlap.

In the acquisition processing SA140, the processing device 210 functionsas the acquirer 210 d. In the acquisition processing SA140, theprocessing device 210 determines whether the size of the overlappingregion is equal to or larger than the threshold. When the size of theoverlapping region is smaller than the threshold, the acquirer 210 dcauses the notifier 210 c to continue displaying the message M1. Whenthe size of the overlapping region is equal to or larger than thethreshold, the acquirer 210 d causes the notifier 210 c to enddisplaying the message M1. At the opportunity when the message M1 is notdisplayed on the display device 420 any longer, the user touches theuser interface image GA5. When detecting the touch on the user interfaceimage GA5, the processing device 210 outputs, to the projector 10, athird signal for instructing the projector 10 to stop projecting thefirst guide image GA1 and writes, in the volatile memory of the storagedevice 230, captured image data representing the captured image GA2 ofthe projection region where the first guide image GA1 is not displayed,that is, reference image data representing a reference image.

In the output processing SA150, the processing device 210 functions asthe output unit 210 e. In the output processing SA150, the processingdevice 210 acquires projection image data representing a projectionimage by causing the user to perform editing for the reference imagerepresented by the reference image data. Subsequently, the processingdevice 210 outputs the projection image data to the projector 10.

As explained above, in this embodiment, what the user of the projector10 is requested is to determine a position of the camera 30 according tothe message M1 and touches the position and to edit a projection imagereferring to the captured image GA2 of the camera 30. In the techniquedisclosed in Patent Literature 1, it is necessary to prepare athree-dimensional model corresponding to the projection target object SCand to coordinate-convert the three-dimensional model into atwo-dimensional image coinciding with a projection apparatus view pointimage. Expertise concerning the three-dimensional model and expertiseconcerning the coordinate conversion are required. In contrast, in thisembodiment, it is unnecessary to prepare a three-dimensional modelcorresponding to the projection target object SC and it is unnecessaryto perform coordinate conversion when editing a projection image.Therefore, according to this embodiment, a user not having expertiseconcerning a three-model and the like is capable of easily performingthe projection mapping.

2. Modifications

The embodiment explained above can be modified as explained below.

(1) In the embodiment explained above, an application example of thepresent disclosure to the projection mapping for realizing decoration inthe room of the user of the projector 10 is explained. However, thepresent disclosure may be applied to projection mapping for realizingcommodity display in a store such as a retail store and may be appliedto projection mapping for realizing performances in a theme park, anevent venue, or the like.

(2) In the embodiment explained above, one projector 10 projects theprojection image onto one projection target object SC. However, aplurality of projectors 10 respectively disposed in different positionsmay project projection images onto one projection target object SC.Since the projection images are projected onto the one projection targetobject SC from the plurality of projectors 10 respectively disposed inthe different positions, projection mapping with increased brightnesscan be realized. Since the projection images are projected onto the oneprojection target object SC from the plurality of projectors 10respectively disposed in the different positions, projection mappingthat reduces shadow as much as possible and can be seen from anywhere in360° can be realized.

(3) When the user interface image GA5 is touched, the acquirer 210 d inthe embodiment explained above outputs the third signal for stoppingdisplaying the first guide image GA1 and, thereafter, acquires thereference image data. However, the output of the third signal may beomitted. In an aspect in which the output of the third image is omitted,the first guide image GA1 is included in the captured image GA2represented by the captured image data output from the camera 30. Theacquirer 210 d may directly write the captured image data representingthe captured image GA2 including the first guide image GA1 in thevolatile memory of the storage device 230 as the reference image data.The first guide image GA1 is included in the reference image in thisaspect. In the aspect in which the output of the third image is omitted,the acquirer 210 d may generate the reference image data by applyingimage processing for erasing or diluting the first guide image GA1 tothe captured image data. Specific examples of the image processing forerasing or diluting the first guide image GA1 include image processingfor replacing a pixel value of a pixel corresponding to the first guideimage GA1 with a pixel value of a peripheral pixel, which is a pixellocated around the pixel and not corresponding to the first guide imageGA1. Other specific examples of the image processing for erasing ordiluting the first guide image GA1 include image processing forreplacing a pixel value of a pixel corresponding to the first guideimage GA1 with an average value of the pixel value of the pixel and apixel value of each of a plurality of peripheral pixels. According tothis aspect, a period from when input operation for instructing imaginguntil when the imaging is actually performed is reduced.

(4) The acquirer 210 d in the embodiment explained above causes thenotifier 210 c to continue the notification for requesting the user tomove the camera 30 when determining that the size of the overlappingregion is smaller than the threshold and ends the notification by thenotifier 210 c when determining that the size of the overlapping regionis equal to or larger than the threshold. However, the acquirer 210 dmay cause the notifier 210 c to perform notification for requesting theuser to perform input for instructing determination of a position of thecamera 30 when determining that the size of the overlapping region isequal to or larger than the threshold. The acquirer 210 d may store thereference image data without waiting for input operation of the userwhen determining that the size of the overlapping region is equal to orlarger than the threshold. According to the latter aspect, it ispossible to automatically store the reference image data when the sizeof the overlapping region is equal to or larger than the threshold.Labor of the user is reduced. In the latter aspect, it is unnecessary todisplay the user interface image GA5. The determination about whetherthe size of the overlapping region is smaller than the threshold is notessential and may be omitted.

(5) The second guide image GA3 in the embodiment explained above is theimage indicating the imaging range of the camera 30. However, in orderto clearly indicate the center portion of the imaging range of thecamera 30, the display controller 210 b may display a third guide imageindicating the center portion of the imaging range of the camera 30 onthe display device 420 instead of or together with the second guideimage GA3. In this aspect, a position in the real space where the centerportion of the imaging range and the center portion of the first guideimage GA1 overlap means a position where a center portion of the thirdguide image in the superimposed image GA4 or the third guide imageitself and the center portion of the first guide image GA1 overlap.

The third guide image may be an image smaller than the second guideimage GA3. Specific examples of the third guide image smaller than thesecond guide image GA3 include the image of the second circle explainedabove, an image of a polygon inscribing or circumscribing the secondcircle, and an image of a plurality of line segments crossing in thecenter of the second circle. By displaying the third guide image smallerthan the second guide image GA3 together with the second guide imageGA3, it is possible to clearly indicate the entire imaging range of thecamera 30 and the center portion of the imaging range to the user. Whenthe third guide image smaller than the second guide image GA3 isdisplayed instead of the second guide image GA3, it is possible toclearly indicate the center portion of the imaging range of the camera30 to the user.

The third guide image may be an image having the same size as the secondguide image GA3. Specific examples of the third guide image having thesame size as the third guide image GA3 include an image including arectangular frame line partitioning the outer edge of the captured imageGA2 and diagonal lines of the frame line and an image of the diagonallines. The third guide image is an example of the third image in thepresent disclosure. In the former aspect, by displaying the third guideimage instead of the second guide image GA3, it is possible to clearlyindicate the entire imaging range of the camera 30 and the centerportion of the imaging range to the user. In the latter aspect, when thethird guide image is displayed together with the second guide image GA3,it is possible to clearly indicate the entire imaging range of thecamera 30 and the center portion of the imaging range to the user. Whenthe third guide image is displayed instead of the second guide imageGA3, it is possible to clearly indicate the center portion of theimaging range of the camera 30 to the user.

In order to clearly indicate the center portion of the first guide imageGA1, an image indicating the center portion of first guide image GA1 maybe included in the first guide image GA1 Specific examples of the imageindicating the center portion of the first guide image GA1 include theimage of the first circle explained above, an image of a polygoninscribing or circumscribing the first circle, and an image of aplurality of line segments crossing in the center of the first circle.

(6) In the embodiment explained above, the output unit 210 e transmitsthe reference image data to the terminal apparatus 40 and starts theimage drawing tool at the opportunity when the reference image data isstored in the volatile memory. However, at the opportunity when thereference image data is stored in the volatile memory, the output unit210 e may display, on the display device 420, a user interface imagesuch as a virtual operation piece for causing the user to input a startof the image drawing tool. In this aspect, the output unit 210 e causesthe terminal apparatus 40 to start the image drawing tool according toinput operation to the user interface image.

(7) The notification processing SA130 in the embodiment explained abovemay be omitted. In an aspect in which the notification processing SA130is omitted, the notifier 210 c may be omitted. This is because aprojection image can be still easily created even if the notificationprocessing SA130 is omitted. When the projector 10 projects the firstguide image GA1 at the opportunity when the projector 10 is turned on,the projection controller 210 a and the projection control processingSA110 can be omitted. It is possible to grasp the imaging range of thecamera 30 from the captured image GA2 and it is also possible to graspthe center portion of the imaging range. Therefore, the superimpositionof the second guide image GA3 may be omitted.

(8) In the embodiment explained above, the camera 30 and the terminalapparatus 40 are the separate apparatuses. However, the camera 30 may beincluded in the terminal apparatus 40. For example, when the terminalapparatus 40 is a smartphone including a camera, the camera of thesmartphone only has to play a role of the camera 30. In the embodiment,the information processing apparatus 20 is an apparatus different fromall of the terminal apparatus 40, the camera 30, and the projector 10.However, the information processing apparatus 20 may be included in anyof the terminal apparatus 40, the camera 30, and the projector 10.

(9) The projection controller 210 a, the display controller 210 b, thenotifier 210 c, the acquirer 210 d, and the output unit 210 e in theembodiment explained above are the software modules. However, any oneof, a plurality of, or all of the projection controller 210 a, thedisplay controller 210 b, the notifier 210 c, the acquirer 210 d, andthe output unit 210 e may be hardware modules such as an ASIC(Application Specific Integrated Circuit). Even if any one of, aplurality of, or all of the projection controller 210 a, the displaycontroller 210 b, the notifier 210 c, the acquirer 210 d, and the outputunit 210 e are hardware modules, the same effects as the effects in theembodiment are achieved.

(10) The program PA may be manufactured alone or may be provided with orwithout charge. Examples of a specific aspect in providing the programPA include an aspect of writing the program PA in a computer-readablestorage medium such as a flash ROM and providing the program PA and anaspect of providing the program PA by downloading the program PA throughan electric communication line such as the Internet. By causing ageneral computer to operate according to the program PA provided bythese aspects, it is possible to cause the computer to execute the imageediting method according to the present disclosure.

3. An Aspect Grasped from at Least One of the Embodiment and theModifications

The present disclosure is not limited to the embodiment and themodifications explained above and can be realized in various aspects ina range not departing from the gist of the present disclosure. Forexample, the present disclosure can also be realized by the followingaspects. Technical features in the embodiment corresponding to technicalfeatures in the aspects described below can be substituted or combinedas appropriate in order to solve a part of all of the problems of thepresent disclosure or achieve a part or all of the effects of thepresent disclosure. Unless the technical features are explained asessential technical features in this specification, the technicalfeatures can be deleted as appropriate.

An image editing method according to an aspect of the present disclosureincludes acquisition processing and output processing. A captured imagecaptured by a camera in a projection region including a projectiontarget object, which is a projection destination of an image from aprojector, a first guide image being displayed in the projection regionby the projector, is acquired. The projection region is a region in areal space where the projector and the projection target object aredisposed. A position of the camera that captures the captured image is aposition in the real space where a center portion of an imaging range ofthe camera and a center portion of the first guide image overlap. In theoutput processing, a projection image is obtained by editing thecaptured image acquired in the acquisition processing. The projectionimage is transmitted to the projector. With the image editing method inthis aspect, even a user not having expertise is capable of easilyperforming projection mapping.

An image editing method according to a more preferable aspect mayfurther include displaying, on a display apparatus, a superimposed imageobtained by superimposing an image obtained by imaging, with the camera,the projection region where the first guide image is projected and athird guide image indicating the center portion of the imaging range.The third guide image is an example of the third image in the presentdisclosure. The position of the camera in the real space is a positionwhere the center portion of the first guide image and a center portionof the third guide image or the third guide image itself overlap in thesuperimposed image. According to this aspect, a user can determine theposition of the camera while grasping a degree of the overlap of thecenter portion of the imaging range of the camera and the center portionof the projection range of the projector from the first guide image andthe second guide image in the superimposed image.

An image editing method in a still more preferable aspect may furtherinclude, when the captured image is acquired, displaying a userinterface image used to invoke an editing function for editing thecaptured image. The second image in this aspect may be obtained byediting the captured image with the editing function. According to thisaspect, the user can invoke, with operation on the user interface image,the editing function for editing the captured image.

An image editing method according to a still more preferable aspect mayinclude outputting a notification for requesting a user to move thecamera to the position in the real space where the center portion of theimaging range and the center portion of the first guide image overlap.According to this aspect, it is possible to request the user to move thecamera to the position in the real space where the center portion of theimaging range of the camera and the center portion of the first guideimage overlap.

An image editing method according to a still more preferable aspect mayinclude: determining whether a size of an overlapping region is equal toor larger than a threshold; and outputting the notification when thesize of the overlapping region is smaller than the threshold. Theoverlapping region means a region where the center portion of theimaging range of the camera and the center portion of the first guideimage overlap in the superimposed image. According to this aspect, whenthe size of the overlapping region is smaller than the threshold, it ispossible to request the user to move the camera to the position in thereal space where the center portion of the imaging range of the cameraand the center portion of the first guide image overlap.

An image editing method according to a still more preferable aspect mayinclude causing the camera to image the projection region when the sizeof the overlapping region is equal to or larger than the threshold.According to this aspect, when the size of the overlapping region isequal to or larger than the threshold, it is possible to cause thecamera to automatically image the projection region. Labor of the useris reduced.

In an image editing method according to a still more preferable aspect,the acquiring the captured image may be acquiring the captured image byimaging the projection region where the first guide image is notdisplayed. According to this aspect, it is possible to edit theprojection image based on the captured image of the projection regionwhere the first guide image is not displayed.

An image editing system according to an aspect of the present disclosureincludes: a camera; and a processing apparatus configured to control thecamera. The processing apparatus executes acquisition processing andoutput processing. In the acquisition processing, the processingapparatus acquires a captured image captured by a camera in a projectionregion including a projection target object, which is a projectiondestination of an image from a projector, a first guide image beingdisplayed in the projection region by the projector. The projectionregion is a region in a real space where the projector and theprojection target object are disposed. A position of the camera thatcaptures the captured image is a position in the real space where acenter portion of an imaging range of the camera and a center portion ofthe first guide image overlap. In the output processing, the processingapparatus transmits, to the projector, projection image datarepresenting a projection image obtained by editing the captured imageacquired in the acquisition processing. With the image editing system inthis aspect, even a user not having expertise is capable of easilyperforming projection mapping.

A non-transitory computer-readable storage medium storing a programaccording to an aspect of the present disclosure causes a computer toexecute acquisition processing and output processing. In the acquisitionprocessing, the computer operating according to this program acquires acaptured image captured by a camera in a projection region including aprojection target object, which is a projection destination of an imagefrom a projector, a first guide image being displayed in the projectionregion by the projector. The projection region is a region in a realspace where the projector and the projection target object are disposed.A position of the camera that captures the captured image is a positionin the real space where a center portion of an imaging range of thecamera and a center portion of the first guide image overlap. In theoutput processing, the computer operating according to the programtransmits, to the projector, projection image data representing aprojection image obtained by editing the captured image acquired in theacquisition processing. With the storage medium storing the program inthis aspect, even a user not having expertise is capable of easilyperforming projection mapping.

What is claimed is:
 1. An image editing method comprising: acquiring acaptured image by imaging, with a camera, from a position in a realspace where a center portion of an imaging range of the camera and acenter portion of a first image overlap, a projection region in the realspace including a projection target object, which is a projectiondestination of an image from a projector, the first image beingdisplayed in the projection region by the projector; and transmitting asecond image obtained by editing the captured image to the projector. 2.The image editing method according to claim 1, further comprisingdisplaying, on a display apparatus, a superimposed image obtained bysuperimposing an image obtained by imaging, with the camera, theprojection region where the first image is projected and a third imageindicating the center portion of the imaging range, wherein the positionis a position where a center portion of the third image and the centerportion of the first image in the superimposed image overlap.
 3. Theimage editing method according to claim 1, further comprising, when thecaptured image is acquired, displaying a user interface image used toinvoke an editing function for editing the captured image, wherein thesecond image is obtained by editing the captured image with the editingfunction.
 4. The image editing method according to claim 1, furthercomprising outputting a notification for requesting a user to move thecamera to the position.
 5. The image editing method according to claim4, further comprising determining whether or not a size of a regionwhere the center portion of the first image and the center portion ofthe imaging range overlap in the superimposed image is smaller than athreshold, wherein the outputting the notification is outputting thenotification when the size of the region is smaller than the threshold.6. The image editing method according to claim 5, further comprisingcausing the camera to image the projection region when the size of theregion is equal to or larger than the threshold.
 7. The image editingmethod according to claim 1, wherein the acquiring the captured image isacquiring the captured image by causing the camera to image theprojection region where the first image is not displayed.
 8. An imageediting system comprising: a camera; and a processing apparatusconfigured to control the camera, the processing apparatus executing:acquiring a captured image by imaging, with a camera, from a position ina real space where a center portion of an imaging range of the cameraand a center portion of a first image overlap, a projection region inthe real space including a projection target object, which is aprojection destination of an image from a projector, the first imagebeing displayed in the projection region by the projector; andtransmitting a second image obtained by editing the captured image tothe projector.
 9. A non-transitory computer-readable storage mediumstoring a program for causing a computer to execute: acquiring acaptured image by imaging, with a camera, from a position in a realspace where a center portion of an imaging range of the camera and acenter portion of a first image overlap, a projection region in the realspace including a projection target object, which is a projectiondestination of an image from a projector, the first image beingdisplayed in the projection region by the projector; and transmitting asecond image obtained by editing the captured image to the projector.